Extracting Multiword Terms from Document Collections

Multiword terms (MWTs) are relevant strings of words in text collections. Once they are automatically extracted, they may be used by an Information Retrieval system, suggesting its users possible conceptual interesting refinements of their information needs. As a matter of fact, these multiword terms point to relevant information, often corresponding to topics and subtopics in the text collection, and maybe quite useful specially for highly refining generic queries. In this paper, we introduce the LocalMaxs algorithm, for automatically extracting multiword terms. This algorithm requires neither empirically suggested thresholds nor complex linguistic filters nor language specific morpho-syntactic rules. These features make this algorithm a suitable approach to extract MWTs from text collections written in any language. Moreover, by introducing the Fair Dispersion Point Normalization concept, we can deal with arbitrarily long MWTs and can compare the results obtained by using different word association measures for MWTs selection. We also introduce our own association measure, the SCP, to work with the LocalMaxs algorithm, and assess the results obtained by comparing it with related statistics-based measures (Specific Mutual Information, Dice, Loglike and  coefficients) used in experiments on a text collection. An Information Retrieval application using our approach is also presented.